MSLTE: multiple self-supervised learning tasks for enhancing EEG emotion recognition.

Objective. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model.Approach. Multiple self-supervised learning tasks are introduced in the proposed model to enhance the generalization of EEG emotion recognition and reduce the overfitting problem to some extent. Firstly, channel masking and frequency masking are introduced to simulate the information loss in certain channels and frequency bands resulting from the instability of EEG, and two self-supervised learning-based feature reconstruction tasks combining masked graph autoencoders (GAE) are constructed to enhance the generalization of the shared encoder. Secondly, to take full advantage of the complementary information contained in these two self-supervised learning tasks to ensure the reliability of feature reconstruction, a weight sharing (WS) mechanism is introduced between the two graph decoders. Thirdly, an adaptive weight multi-task loss (AWML) strategy based on homoscedastic uncertainty is adopted to combine the supervised learning loss and the two self-supervised learning losses to enhance the performance further.Main results. Experimental results on SEED, SEED-V, and DEAP datasets demonstrate that: (i) Generally, the proposed model achieves higher averaged emotion classification accuracy than various baselines included in both subject-dependent and subject-independent scenarios. (ii) Each key module contributes to the performance enhancement of the proposed model. (iii) It achieves higher training efficiency, and significantly lower model size and computational complexity than the state-of-the-art (SOTA) multi-task-based model. (iv) The performances of the proposed model are less influenced by the key parameters.Significance. The introduction of the self-supervised learning task helps to enhance the generalization of the EEG emotion recognition model and eliminate overfitting to some extent, which can be modified to be applied in other EEG-based classification tasks.

CAS(ME)3: A Third Generation Facial Spontaneous Micro-Expression Database With Depth Information and High Ecological Validity.

Micro-expression (ME) is a significant non-verbal communication clue that reveals one person's genuine emotional state. The development of micro-expression analysis (MEA) has just gained attention in the last decade. However, the small sample size problem constrains the use of deep learning on MEA. Besides, ME samples distribute in six different databases, leading to database bias. Moreover, the ME database development is complicated. In this article, we introduce a large-scale spontaneous ME database: CAS(ME) 3. The contribution of this article is summarized as follows: (1) CAS(ME) 3 offers around 80 hours of videos with over 8,000,000 frames, including manually labeled 1,109 MEs and 3,490 macro-expressions. Such a large sample size allows effective MEA method validation while avoiding database bias. (2) Inspired by psychological experiments, CAS(ME) 3 provides the depth information as an additional modality unprecedentedly, contributing to multi-modal MEA. (3) For the first time, CAS(ME) 3 elicits ME with high ecological validity using the mock crime paradigm, along with physiological and voice signals, contributing to practical MEA. (4) Besides, CAS(ME) 3 provides 1,508 unlabeled videos with more than 4,000,000 frames, i.e., a data platform for unsupervised MEA methods. (5) Finally, we demonstrate the effectiveness of depth information by the proposed depth flow algorithm and RGB-D information.

Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images.

Machine-assisted pathological recognition has been focused on supervised learning (SL) that suffers from a significant annotation bottleneck. We propose a semi-supervised learning (SSL) method based on the mean teacher architecture using 13,111 whole slide images of colorectal cancer from 8803 subjects from 13 independent centers. SSL (~3150 labeled, ~40,950 unlabeled; ~6300 labeled, ~37,800 unlabeled patches) performs significantly better than the SL. No significant difference is found between SSL (~6300 labeled, ~37,800 unlabeled) and SL (~44,100 labeled) at patch-level diagnoses (area under the curve (AUC): 0.980 ± 0.014 vs. 0.987 ± 0.008, P value = 0.134) and patient-level diagnoses (AUC: 0.974 ± 0.013 vs. 0.980 ± 0.010, P value = 0.117), which is close to human pathologists (average AUC: 0.969). The evaluation on 15,000 lung and 294,912 lymph node images also confirm SSL can achieve similar performance as that of SL with massive annotations. SSL dramatically reduces the annotations, which has great potential to effectively build expert-level pathological artificial intelligence platforms in practice.

Manifold adversarial training for supervised and semi-supervised learning.

We propose a new regularization method for deep learning based on the manifold adversarial training (MAT). Unlike previous regularization and adversarial training methods, MAT further considers the local manifold of latent representations. Specifically, MAT manages to build an adversarial framework based on how the worst perturbation could affect the statistical manifold in the latent space rather than the output space. Particularly, a latent feature space with the Gaussian Mixture Model (GMM) is first derived in a deep neural network. We then define the smoothness by the largest variation of Gaussian mixtures when a local perturbation is given around the input data point. On one hand, the perturbations are added in the way that would rough the statistical manifold of the latent space the worst. On the other hand, the model is trained to promote the manifold smoothness the most in the latent space. Importantly, since the latent space is more informative than the output space, the proposed MAT can learn a more robust and compact data representation, leading to further performance improvement. The proposed MAT is important in that it can be considered as a superset of one recently-proposed discriminative feature learning approach called center loss. We conduct a series of experiments in both supervised and semi-supervised learning on four benchmark data sets, showing that the proposed MAT can achieve remarkable performance, much better than those of the state-of-the-art approaches. In addition, we present a series of visualization which could generate further understanding or explanation on adversarial examples.

Supervised and semi-supervised probabilistic learning with deep neural networks for concurrent process-quality monitoring.

Concurrent process-quality monitoring helps discover quality-relevant process anomalies and quality-irrelevant process anomalies. It especially works well in chemical plants with faults that cause quality problems. Traditional monitoring strategies are limitedly applied in chemical plants because quality targets in training data are insufficient. It is hard for inflexible models to fully capture the strongly nonlinear process-quality correlations. Also, deterministic models are mapped from process variables to qualities without any consideration of uncertainties. Simultaneously, a slow sampling rate for quality variables is ubiquitous in chemical plants since a product quality test is often time-consuming and expensive. Motivated by these limitations, this paper proposes a new concurrent process-quality monitoring scheme based on a probabilistic generative deep learning model developed from variational autoencoder. The supervised model is firstly developed and then the semi-supervised version is extended to solve the issue of missing targets. Especially, the semi-supervised learning algorithm is accomplished with an optimal parameter estimation in the light of maximum likelihood principle and no any hyperparameters are introduced. Two case studies validate that the proposed method effectively outperforms the other comparative methods in concurrent process-quality monitoring.

Using supervised machine learning on neuropsychological data to distinguish OCD patients with and without sensory phenomena from healthy controls.

OBJECTIVES: While theoretical models link obsessive-compulsive disorder (OCD) with executive function deficits, empirical findings from the neuropsychological literature remain mixed. These inconsistencies are likely exacerbated by the challenge of high-dimensional data (i.e., many variables per subject), which is common across neuropsychological paradigms and necessitates analytical advances. More unique to OCD is the heterogeneity of symptom presentations, each of which may relate to distinct neuropsychological features. While researchers have traditionally attempted to account for this heterogeneity using a symptom-based approach, an alternative involves focusing on underlying symptom motivations. Although the most studied symptom motivation involves fear of harmful events, 60-70% of patients also experience sensory phenomena, consisting of uncomfortable sensations or perceptions that drive compulsions. Sensory phenomena have received limited attention in the neuropsychological literature, despite evidence that symptoms motivated by these experiences may relate to distinct cognitive processes. METHODS: Here, we used a supervised machine learning approach to characterize neuropsychological processes in OCD, accounting for sensory phenomena. RESULTS: Compared to logistic regression and other algorithms, random forest best differentiated healthy controls (n = 59; balanced accuracy = .70), patients with sensory phenomena (n = 29; balanced accuracy = .59), and patients without sensory phenomena (n = 46; balanced accuracy = .62). Decision-making best distinguished between groups based on sensory phenomena, and among the patient subsample, those without sensory phenomena uniquely displayed greater risk sensitivity compared to healthy controls (d = .07, p = .008). CONCLUSIONS: Results suggest that different cognitive profiles may characterize patients motivated by distinct drives. The superior performance and generalizability of the newer algorithms highlights the utility of considering multiple analytic approaches when faced with complex data. PRACTITIONER POINTS: Practitioners should be aware that sensory phenomena are common experiences among patients with OCD. OCD patients with sensory phenomena may be distinguished from those without based on neuropsychological processes.

Supervised learning in spiking neural networks: A review of algorithms and evaluations.

As a new brain-inspired computational model of the artificial neural network, a spiking neural network encodes and processes neural information through precisely timed spike trains. Spiking neural networks are composed of biologically plausible spiking neurons, which have become suitable tools for processing complex temporal or spatiotemporal information. However, because of their intricately discontinuous and implicit nonlinear mechanisms, the formulation of efficient supervised learning algorithms for spiking neural networks is difficult, and has become an important problem in this research field. This article presents a comprehensive review of supervised learning algorithms for spiking neural networks and evaluates them qualitatively and quantitatively. First, a comparison between spiking neural networks and traditional artificial neural networks is provided. The general framework and some related theories of supervised learning for spiking neural networks are then introduced. Furthermore, the state-of-the-art supervised learning algorithms in recent years are reviewed from the perspectives of applicability to spiking neural network architecture and the inherent mechanisms of supervised learning algorithms. A performance comparison of spike train learning of some representative algorithms is also made. In addition, we provide five qualitative performance evaluation criteria for supervised learning algorithms for spiking neural networks and further present a new taxonomy for supervised learning algorithms depending on these five performance evaluation criteria. Finally, some future research directions in this research field are outlined.

Single-modal and multi-modal false arrhythmia alarm reduction using attention-based convolutional and recurrent neural networks.

This study proposes a deep learning model that effectively suppresses the false alarms in the intensive care units (ICUs) without ignoring the true alarms using single- and multi- modal biosignals. Most of the current work in the literature are either rule-based methods, requiring prior knowledge of arrhythmia analysis to build rules, or classical machine learning approaches, depending on hand-engineered features. In this work, we apply convolutional neural networks to automatically extract time-invariant features, an attention mechanism to put more emphasis on the important regions of the segmented input signal(s) that are more likely to contribute to an alarm, and long short-term memory units to capture the temporal information presented in the signal segments. We trained our method efficiently using a two-step training algorithm (i.e., pre-training and fine-tuning the proposed network) on the dataset provided by the PhysioNet computing in cardiology challenge 2015. The evaluation results demonstrate that the proposed method obtains better results compared to other existing algorithms for the false alarm reduction task in ICUs. The proposed method achieves a sensitivity of 93.88% and a specificity of 92.05% for the alarm classification, considering three different signals. In addition, our experiments for 5 separate alarm types leads significant results, where we just consider a single-lead ECG (e.g., a sensitivity of 90.71%, a specificity of 88.30%, an AUC of 89.51 for alarm type of Ventricular Tachycardia arrhythmia).

Recommendations and future directions for supervised machine learning in psychiatry.

Machine learning methods hold promise for personalized care in psychiatry, demonstrating the potential to tailor treatment decisions and stratify patients into clinically meaningful taxonomies. Subsequently, publication counts applying machine learning methods have risen, with different data modalities, mathematically distinct models, and samples of varying size being used to train and test models with the promise of clinical translation. Consequently, and in part due to the preliminary nature of such works, many studies have reported largely varying degrees of accuracy, raising concerns over systematic overestimation and methodological inconsistencies. Furthermore, a lack of procedural evaluation guidelines for non-expert medical professionals and funding bodies leaves many in the field with no means to systematically evaluate the claims, maturity, and clinical readiness of a project. Given the potential of machine learning methods to transform patient care, albeit, contingent on the rigor of employed methods and their dissemination, we deem it necessary to provide a review of current methods, recommendations, and future directions for applied machine learning in psychiatry. In this review we will cover issues of best practice for model training and evaluation, sources of systematic error and overestimation, model explainability vs. trust, the clinical implementation of AI systems, and finally, future directions for our field.

An optimal time interval of input spikes involved in synaptic adjustment of spike sequence learning.

The supervised learning methods for spiking neurons based on temporal encoding are important foundation for the development of spiking neural networks. During the learning process, the synaptic weights of a spiking neuron are adjusted to make the neuron emit a specific spike train. Because various learning methods use the information of input spikes to calculate the adjustment of synaptic weights, how many input spikes participated in the calculation is a critical factor that can influence learning performance. This paper chooses an important category of learning methods as the research object to study the factor. The input spikes participated in weight adjustment are contained in a time interval. An optimal time interval that contains the most appropriate number of input spikes is proposed based on the characteristic of the category of learning methods. The length of the optimal time interval is determined by comprehensive consideration of desired and actual output spikes. The results of a lot of experiments show that the optimal time interval can obtain the highest learning performance under various experimental settings. If other time intervals are longer than the optimal time interval, an overlapping problem of input spikes will occur and the learning performance will decline a lot. The learning accuracy of the optimal time interval can be about 55% higher than the learning accuracy of an other longer time interval. If other time intervals are shorter than the optimal time interval, the input spikes contained in them will be insufficient to adjust synaptic weights and the learning performance will also decline. The learning accuracy of the optimal time interval can be about 8% higher than the learning accuracy of an other shorter time interval. In addition, the optimal time interval can also improve the generalization ability and pattern storage capability of the category of learning methods.

A supervised learning approach for diffusion MRI quality control with minimal training data.

Quality control (QC) is a fundamental component of any study. Diffusion MRI has unique challenges that make manual QC particularly difficult, including a greater number of artefacts than other MR modalities and a greater volume of data. The gold standard is manual inspection of the data, but this process is time-consuming and subjective. Recently supervised learning approaches based on convolutional neural networks have been shown to be competitive with manual inspection. A drawback of these approaches is they still require a manually labelled dataset for training, which is itself time-consuming to produce and still introduces an element of subjectivity. In this work we demonstrate the need for manual labelling can be greatly reduced by training on simulated data, and using a small amount of labelled data for a final calibration step. We demonstrate its potential for the detection of severe movement artefacts, and compare performance to a classifier trained on manually-labelled real data.

Semi-supervised machine learning for automated species identification by collagen peptide mass fingerprinting.

BACKGROUND: Biomolecular methods for species identification are increasingly being utilised in the study of changing environments, both at the microscopic and macroscopic levels. High-throughput peptide mass fingerprinting has been largely applied to bacterial identification, but increasingly used to identify archaeological and palaeontological skeletal material to yield information on past environments and human-animal interaction. However, as applications move away from predominantly domesticate and the more abundant wild fauna to a much wider range of less common taxa that do not yet have genetically-derived sequence information, robust methods of species identification and biomarker selection need to be determined. RESULTS: Here we developed a supervised machine learning algorithm for classifying the species of ancient remains based on collagen fingerprinting. The aim was to minimise requirements on prior knowledge of known species while yielding satisfactory sensitivity and specificity. The algorithm uses iterations of a modified random forest classifier with a similarity scoring system to expand its identified samples. We tested it on a set of 6805 spectra and found that a high level of accuracy can be achieved with a training set of five identified specimens per taxon. CONCLUSIONS: This method consistently achieves higher accuracy than two-dimensional principal component analysis and similar accuracy with hierarchical clustering using optimised parameters, which greatly reduces requirements for human input. Within the vertebrata, we demonstrate that this method was able to achieve the taxonomic resolution of family or sub-family level whereas the genus- or species-level identification may require manual interpretation or further experiments. In addition, it also identifies additional species biomarkers than those previously published.

Protein-protein interaction site predictions with minimum covariance determinant and Mahalanobis distance.

Protein-protein interaction site (PPIS) prediction must deal with the diversity of interaction sites that limits their prediction accuracy. Use of proteins with unknown or unidentified interactions can also lead to missing interfaces. Such data errors are often brought into the training dataset. In response to these two problems, we used the minimum covariance determinant (MCD) method to refine the training data to build a predictor with better performance, utilizing its ability of removing outliers. In order to predict test data in practice, a method based on Mahalanobis distance was devised to select proper test data as input for the predictor. With leave-one-validation and independent test, after the Mahalanobis distance screening, our method achieved higher performance according to Matthews correlation coefficient (MCC), although only a part of test data could be predicted. These results indicate that data refinement is an efficient approach to improve protein-protein interaction site prediction. By further optimizing our method, it is hopeful to develop predictors of better performance and wide range of application.

Comparison of machine-learning algorithms to build a predictive model for detecting undiagnosed diabetes - ELSA-Brasil: accuracy study.

CONTEXT AND OBJECTIVE:: Type 2 diabetes is a chronic disease associated with a wide range of serious health complications that have a major impact on overall health. The aims here were to develop and validate predictive models for detecting undiagnosed diabetes using data from the Longitudinal Study of Adult Health (ELSA-Brasil) and to compare the performance of different machine-learning algorithms in this task. DESIGN AND SETTING:: Comparison of machine-learning algorithms to develop predictive models using data from ELSA-Brasil. METHODS:: After selecting a subset of 27 candidate variables from the literature, models were built and validated in four sequential steps: (i) parameter tuning with tenfold cross-validation, repeated three times; (ii) automatic variable selection using forward selection, a wrapper strategy with four different machine-learning algorithms and tenfold cross-validation (repeated three times), to evaluate each subset of variables; (iii) error estimation of model parameters with tenfold cross-validation, repeated ten times; and (iv) generalization testing on an independent dataset. The models were created with the following machine-learning algorithms: logistic regression, artificial neural network, naïve Bayes, K-nearest neighbor and random forest. RESULTS:: The best models were created using artificial neural networks and logistic regression. -These achieved mean areas under the curve of, respectively, 75.24% and 74.98% in the error estimation step and 74.17% and 74.41% in the generalization testing step. CONCLUSION:: Most of the predictive models produced similar results, and demonstrated the feasibility of identifying individuals with highest probability of having undiagnosed diabetes, through easily-obtained clinical data.

Comparison of machine-learning algorithms to build a predictive model for detecting undiagnosed diabetes - ELSA-Brasil: accuracy study / Comparação de algoritmos de aprendizagem de máquina para construir um modelo preditivo para detecção de diabetes não diagnosticada - ELSA-Brasil: estudo de acurácia

ABSTRACT CONTEXT AND OBJECTIVE: Type 2 diabetes is a chronic disease associated with a wide range of serious health complications that have a major impact on overall health. The aims here were to develop and validate predictive models for detecting undiagnosed diabetes using data from the Longitudinal Study of Adult Health (ELSA-Brasil) and to compare the performance of different machine-learning algorithms in this task. DESIGN AND SETTING: Comparison of machine-learning algorithms to develop predictive models using data from ELSA-Brasil. METHODS: After selecting a subset of 27 candidate variables from the literature, models were built and validated in four sequential steps: (i) parameter tuning with tenfold cross-validation, repeated three times; (ii) automatic variable selection using forward selection, a wrapper strategy with four different machine-learning algorithms and tenfold cross-validation (repeated three times), to evaluate each subset of variables; (iii) error estimation of model parameters with tenfold cross-validation, repeated ten times; and (iv) generalization testing on an independent dataset. The models were created with the following machine-learning algorithms: logistic regression, artificial neural network, naïve Bayes, K-nearest neighbor and random forest. RESULTS: The best models were created using artificial neural networks and logistic regression. ­These achieved mean areas under the curve of, respectively, 75.24% and 74.98% in the error estimation step and 74.17% and 74.41% in the generalization testing step. CONCLUSION: Most of the predictive models produced similar results, and demonstrated the feasibility of identifying individuals with highest probability of having undiagnosed diabetes, through easily-obtained clinical data.

RESUMO CONTEXTO E OBJETIVO: Diabetes tipo 2 é uma doença crônica associada a graves complicações de saúde, causando grande impacto na saúde global. O objetivo foi desenvolver e validar modelos preditivos para detectar diabetes não diagnosticada utilizando dados do Estudo Longitudinal de Saúde do Adulto (ELSA-Brasil) e comparar o desempenho de diferentes algoritmos de aprendizagem de máquina. TIPO DE ESTUDO E LOCAL: Comparação de algoritmos de aprendizagem de máquina para o desenvolvimento de modelos preditivos utilizando dados do ELSA-Brasil. MÉTODOS: Após selecionar 27 variáveis candidatas a partir da literatura, modelos foram construídos e validados em 4 etapas sequenciais: (i) afinação de parâmetros com validação cruzada (10-fold cross-validation); (ii) seleção automática de variáveis utilizando seleção progressiva, estratégia "wrapper" com quatro algoritmos de aprendizagem de máquina distintos e validação cruzada para avaliar cada subconjunto de variáveis; (iii) estimação de erros dos parâmetros dos modelos com validação cruzada; e (iv) teste de generalização em um conjunto de dados independente. Os modelos foram criados com os seguintes algoritmos de aprendizagem de máquina: regressão logística, redes neurais artificiais, naïve Bayes, K vizinhos mais próximos e floresta aleatória. RESULTADOS: Os melhores modelos foram criados utilizando redes neurais artificiais e regressão logística alcançando, respectivamente, 75,24% e 74,98% de média de área sob a curva na etapa de estimação de erros e 74,17% e 74,41% na etapa de teste de generalização. CONCLUSÃO: A maioria dos modelos preditivos produziu resultados semelhantes e demonstrou a viabilidade de identificar aqueles com maior probabilidade de ter diabetes não diagnosticada com dados clínicos facilmente obtidos.

Accuracy of latent-variable estimation in Bayesian semi-supervised learning.

Hierarchical probabilistic models, such as Gaussian mixture models, are widely used for unsupervised learning tasks. These models consist of observable and latent variables, which represent the observable data and the underlying data-generation process, respectively. Unsupervised learning tasks, such as cluster analysis, are regarded as estimations of latent variables based on the observable ones. The estimation of latent variables in semi-supervised learning, where some labels are observed, will be more precise than that in unsupervised, and one of the concerns is to clarify the effect of the labeled data. However, there has not been sufficient theoretical analysis of the accuracy of the estimation of latent variables. In a previous study, a distribution-based error function was formulated, and its asymptotic form was calculated for unsupervised learning with generative models. It has been shown that, for the estimation of latent variables, the Bayes method is more accurate than the maximum-likelihood method. The present paper reveals the asymptotic forms of the error function in Bayesian semi-supervised learning for both discriminative and generative models. The results show that the generative model, which uses all of the given data, performs better when the model is well specified.